Top-k projection

ABSTRACT

In an example, a top-k function is associated with a top-k projection for a data storage system. Input data to be loaded into the data storage system is divided and ranked according to the top-k function and stored in the top-k projection.

BACKGROUND

In traditional database architectures, data is stored in databasetables. Additionally, structures such as indexes are created forimproved query performance.

When a query is received, the database tables are searched for matchingresults. Instead of having to read every record in a database table tosearch for matches, which can take a significant amount of timeespecially for large tables, indexes may be used to reduce the searchtime for identifying records matching the query.

Indexing is a way of sorting records on a field or multiple fields.Creating an index on a field in a database table creates another datastructure which may hold, for each field value, a pointer that points toits related record. The index is sorted, so a faster search can beperformed on the index, which typically does not require accessing everyrecord stored in the database table.

BRIEF DESCRIPTION OF THE DRAWINGS

Features of the present disclosure are illustrated by way of example andnot limited in the following figure(s), in which like numerals indicatelike elements, in which:

FIG. 1 shows a block diagram of a data storage system, according to anexample of the present disclosure;

FIG. 2 shows a block diagram of a computer platform for the data storagesystem, according to an example of the present disclosure;

FIG. 3 shows examples of loading data into super projections and top-kprojections;

FIG. 4 shows a flow diagram of a method to load data into a top-kprojection, according to an example of the present disclosure;

FIG. 5 shows a flow diagram of a method to create and populate a top-kprojection, according to an example of the present disclosure; and

FIG. 6 shows a flow diagram of a method to query a top-k projection,according to an example of the present disclosure.

DETAILED DESCRIPTION

For simplicity and illustrative purposes, the present disclosure isdescribed by referring mainly to an example thereof. In the followingdescription, numerous specific details are set forth in order to providea thorough understanding of the present disclosure. It will be readilyapparent however, that the present disclosure may be practiced withoutlimitation to these specific details. In other instances, some methodsand structures have not been described in detail so as not tounnecessarily obscure the present disclosure. As used herein, the terms“a” and “an” are intended to denote at least one of a particularelement, the term “includes” means includes but not limited to, the term“including” means including but not limited to, and the term “based on”means based at least in part on.

According to an example, a database storage system can create and storedata in top-k projections, and run queries on top-k projections. Thedatabase storage system performs database functions, such as storingdata, running queries on the stored data, etc. The top-k projections andother projections used by database storage system may be used instead ofindexes to optimize query execution.

The data storage system stores data in projections. A projection is anoptimized collection of table columns that provide physical storage fordata. A projection is a data structure that can contain one, some or allof the columns of one or more database tables and the projectionscontain the actual data of the columns, which may be compressed. Aprojection that contains all of the columns of a database table iscalled a super projection. Even though projections are used, thedatabase schema, e.g., identifying fields (i.e., columns), foreign keys,primary keys, etc., is supported by the projections. Also, the databasestorage system can execute standard database commands but the executionof the commands is performed on the projections rather than on typicaldatabase tables.

As indicated above, a projection is an optimized collection of tablecolumns. Optimization refers to the improvement of performance ofdatabase function, such as storage, query execution, etc. For example,columns can be selected for a projection to improve the speed ofexecuting a query.

A top-k projection is a projection that stores pre-computed top-k rowsof values from each partition of rows of a table that may include manypartitions of rows. In many application domains, users may be interestedin a certain number of the highest ranking (i.e., top-k) rows from apotentially huge data space. Accordingly, the top-k function may returna k number of the highest ranking rows in response to a user query. Thehighest ranking rows may be identified from each partition of rows basedon a score or rank assigned to the rows by a scoring function, forinstance.

The data storage system supports the following functions for top-kprojections: creating a top-k projection; insert and copy (bulk load)data into a top-k projection; select data from a top-k projection;update data of a top-k projection; delete data from a top-k projectionand drop a partition of a top-k projection; merge data of a top-kprojection that is stored in different physical containers; refresh datafor a top-k projection; recover data for a top-k projection after a nodeof a cluster is down and then brought back up; rebalance data of a top-kprojection when new nodes are added to a cluster of nodes of the datastorage system; and backup and recovery of data.

A top-k projection can store results of a top-k function in a column ofthe top-k projection. The top-k function may be based on a predeterminedquery that is commonly executed for a user. The results of the top-kfunction can be stored in the column prior to receiving thepredetermined query. Accordingly, when a query is received that is basedon the top-k function, the results can be quickly provided to the userfrom the top-k projection. Thus, query execution performance isimproved, i.e., faster.

The data storage system for example does not use indexes and traditionaldatabase tables to store data and run queries, such as in a conventionaldatabase management system. Instead, the projections, including thetop-k projections, are used to store data and run queries. For example,instead of storing a database table including rows of data, the datastorage system stores the data in a super projection and otherprojections, and executes queries on the projections. Execution ofqueries on projections may be faster than using indexes and databasetables to execute the queries if the projections are optimized for thequeries. Also, the data storage system does not need to waste storage onindexes.

FIG. 1 is an example of a data storage system 100 that uses projections,including top-k projections, to execute database functions, includingstoring data and running queries. The data storage system 100 includes aload management module 110, a projection manager 120 and a query engine130. The data storage system 100 also includes data storage 150 to storeprojections including data 140, a, projection catalog, and otherinformation used by the data storage system 100. The data storage 150may include disks, memory and/or other types of storage mediums.

Computers or users may provide data 140 for storage in projections inthe data storage system 100, provide projection input 141 for creatingor modifying projections, send queries 142, and get query results 143from the data storage system 100. For example, the load managementmodule 110 receives the data 140 and stores the data in thecorresponding projections. A traditional relational database has atleast one table including columns for storing data. One or moreprojections, including columns of the table, instead of traditionaldatabase tables and indexes. However, information about the databaseschema, tables for the schema, projections, etc., may be stored in thedata storage system 100 so the data storage system 100 can identify andstore data in the corresponding projections and run queries on theprojections that optimize query performance.

When an instruction or command is received to load the data 140 into atable, the load management module 110 stores the data in theprojections. For example, the data storage system 100 supportsconventional database languages, such as structured query language(SQL). The load management module 110 receives a SQL load command toload data into a table, such as INSERT (row by row) or COPY (bulk load aset of rows) command, and loads the data in the projections. Receiveddata is loaded into a super projection, which includes all the columnsof the database table, and is loaded into other projections. The loadmanagement module 110 directs the appropriate data of columns to theright target projections. For example, a projection catalog stored inthe data storage 150 stores information about each projection in thedata storage system, including the columns of each projection. The loadmanagement module 110 directs the appropriate data of columns to theright target projections based on descriptions of the projections in theprojection catalog.

For example, the load management module 110 performs expressionevaluation on received load commands through the operator ExprEval toload data into the projections. For normal projections, the ExprEvaloperator directs the columns to the right order. For top-k projection,the ExprEval operator may also computes expressions and then passes theexpressions to a top-k operator, which retrieves the top-k rows from apartition of rows of a table and passes the top-k rows to the targettop-k projection. In one example, the top-k function is calculated onthe received data to be inserted or bulk-loaded into the projectionsrather than on both the received data and the data already in theprojections. For example, the existing data in the top-k projection isnot combined with the received data to be inserted or bulk-loaded intothe top-k projection. Thus, the top-k projection may store rows that arenot in the top-k; such rows are filtered out when data is queried.

The query engine 130 receives queries 142 which may be in SQL andexecutes the queries on the stored projections to return query results142. For example, a user or system submits a query to the data storagesystem 100. The query engine 130 parses the query and chooses the bestprojection(s) for the query by referencing the projection catalog. Forexample, the projection catalog 300 stores information about theprojections and their columns, including user, schema, table, andprojection information. A projection may be identified that alreadystores the query results in a column. For example, a particular top-kprojection has columns storing results for a top-k function. If thequery is requesting the same top-k data stored in the column, the queryengine 130 runs the query on the top-k projection. If no projection isoptimized for the received query, the query may be executed on the superprojection. The query results are returned to the system or end userthat sent the query.

The projection manager 120 creates and manages projections and storesinformation about the projections in the projection catalog. Projectioninput 141 may be provided by a user or system. The projection input 141may include commands for creating or modifying projections. Theprojection manager 120 creates or modifies the projections according tothe commands, and updates the projection catalog if needed.

FIG. 2 illustrates a block diagram of computer 200, including hardwareand a non-transitory computer readable medium that may be used as aplatform for the data storage system 100.

For example, the computer 200 includes a processor 202, a data storagedevice 204, and an input/output interface 206. In one example, thecomputer is a server but other types of computers may be used. Also, thecomponents are shown in a single computer as an example and in otherexamples the components may exist on multiple computers and thecomponents may comprise multiple processors, data storage devices,interfaces, etc.

The data storage device 204 may include a hard disk, memory, etc. Thedata storage 204 may store any data used by the data storage system 100.The processor 202 may be a microprocessor, a micro-controller, anapplication specific integrated circuit (ASIC), field programmable gatearray (FPGA), or other type of circuit to perform various processingfunctions.

In one example, the data storage system 100 includes machine readableinstructions stored on a non-transitory computer readable medium, suchas the data storage device 204, and executed by the processor 202 toperform the functions of the data storage system 100. For example, theload management module 110, projection manager 120 and query engine 130may include machine readable instructions stored on the data storagedevice 204 as shown in FIG. 2. In another example, the data storagesystem 100 includes a customized circuit, such as anapplication-specific integrated circuit (ASIC) or field-programmablegate array (FPGA) to perform a function or multiple functions of theload management module 110, projection manager 120 and query engine 130.

The input/output (I/O) interface 206 comprises a hardware and/or asoftware interface. The I/O interface 206 may be a network interfaceconnected to a network, such as the Internet, a local area network, etc.The data storage system 100 may receive data and user-input through theI/O interface 206. The data storage system 100 may be connected to thedata storage 150, which may be provided on the computer 200 or on aseparate computer.

The computer 200 may be a node of a distributed data storage system. Forexample, the computer 200 may be part of a cluster of nodes thatservices queries and provide data storage for multiple users or systems,and the nodes may communicate with each other to service queries andstore data. The computer 200 and nodes 201-204 are shown as connectedvia network 220. Nodes may provide data redundancy to minimize down timein case of a node failure.

The data storage system 100 may receive an instruction (e.g., SQLstatement) to create a table, tab1, with columns of different datatypes. An example of the statement is shown below. The columns in thetable are id, f, i, c, v, and d. The table is partitioned on the year ofcolumn d.

Table Create Statement CREATE TABLE  tab1  (id  INT PRIMARY KEY,  f FLOAT,  i  INT,  c  CHAR,  v  VARCHAR,  d  TIMESTAMP not null)PARTITION BY EXTRACT (YEAR FROM d);

The projection manager 120, shown in FIG. 1, creates a super projectionfor the table that includes all the columns. A user or system may setparameters for sorting, segmenting and storing the super projection inparticular nodes. Below are examples of statements for creating twosuper projections, sp1 and sp2, segmented on columns (i) and (c, i)respectively. SELECT indicates the columns. All the columns are providedin the super projection but the order of the columns in the projectionmay be specified by listing the columns. FROM indicates the table havingthe columns. ORDER BY may be used to specify the sort field for sortingthe rows in the projection. Partitioned rows may be provided in theirown container per partition. For example, if partitioned on year, allrows for a year may be provided in a corresponding storage container.

Super Projection Create Statements

CREATE PROJECTION sp1 AS

SELECT i, c, v, d, f, id FROM tab1 ORDER BY id SEGMENTED BY HASH(i) ALLNODES;

CREATE PROJECTION sp2 AS

SELECT * FROM tab1 SEGMENTED BY HASH(c, i) ALL NODES KSAFE1;

3. Examples of Top-K Query, Top-K Projection Creation and Population

The data storage system 100 may receive a top-k query or function (e.g.,an SQL statement) to retrieve the top-k rows from a partition (i.e.,group) of rows of a table tab 1, that may include many partitions ofrows. An example of the statement is shown below. The following top-kquery (Q1) selects values f, i, d (i.e., SELECT) for the one (i.e.,LIMIT 1) most recent d (i.e., ORDER BY) for each partition of i (i.e.PARTITION BY) from the table tab1 (i.e. FROM). In other words, top-kquery Q1 retrieves value f of the oldest timestamp d for each partitioni from the table tab1.

Top-K Query Statement

Q1: SELECT f, i, d FROM tab1 LIMIT 1 OVER (PARTITION BY i ORDER BY d);

According to an example, table tab1 may include the data shown below inTABLE 1 after a first load of input data.

TABLE 1 id f i C v D 1 10 100 ‘c’ ‘cat’ ‘2014-01-20’ 2 11 101 ‘d’ ‘dog’‘2014-01-15’ 3 15 100 ‘c’ ‘cow’ '2014-01-15' 4 17 101 ‘b’ ‘bat’ 2012-11-01’

If the top-k query Q1 is run on the input data shown in TABLE 1,intermediate data representing different partitions of same values incolumn i is shown below in TABLE 2. As shown in TABLE 2, the data of thethree columns (f, i, d) are partitioned or divided by partitions of samevalues in column i (e.g., 100 and 101), and each partition is sorted byd.

TABLE 2 f i d 15 100 ‘2014-01-15’ 10 100 ‘2014-01-20’ 17 101 2012-11-01’ 11 101 ‘2014-01-15’

The result of the top-k query Q1 includes one top row for eachpartition. For instance, the top-k query Q1 retrieves one top row of theoldest timestamp d for each partition l (e.g., 100 and 101). Theresulting data is shown below in TABLE 3.

TABLE 3 f i d 15 100 ‘2014-01-15’ 17 101  2012-11-01’

According to an example, a top-k projection may be created and populatedwith pre-computed top-k result data for each partition of rows tabletab1 using the top-k query statement discussed above. Particularly, thetop-k projection that stores the result data specified in the top-kquery Q1 may be defined as follows:

Top-K Projection Creation Statement

CREATE PROJECTION projection_name AS [Top-K Query]

In other words, the top-k projection creation statement may be furtherexpanded as follows:

CREATE PROJECTION projection_name AS

SELECT expressions FROM table_name

LIMIT a_number OVER (PARTITION BY expressions ORDER BY expressions);

Accordingly, using the above discussed examples of table tab1 and top-kquery Q1, the top-k projection named topk1 may be defined by thefollowing statement:

CREATE PROJECTION topk1 AS

SELECT i, d, f FROM tab1 LIMIT 1 OVER (PARTITION BY i ORDER BY d);

The top-k query or function specified for projection topk1 createscolumns for the top-k projection to store the results of the top-k querypartitioned by the column(s) specified in the creation statement. In anexample, the top-k projection is automatically partitioned onexpressions in the PARTITION BY clause, and is automatically sorted onexpressions in the PARTITION BY clause. For example, the expressions ofthe ORDER BY clause of the OVER clause is used to order each individualpartition defined in the PARTITION BY clause.

Also, in an example, if the expression in the PARTITION BY clause of thetable of the projection is not included in the top-k projection, it willbe added automatically into the top-k projection when the projection iscreated but invisible to the users. For example, table tab1 ispartitioned by the year of column d. Thus, the top-k projectionstatement CREATE PROJECTION topk1 AS SELECT i, d, f FROM tab1 LIMIT 1OVER (PARTITION BY i ORDER BY d) may be considered as the followingtop-k projection statement CREATE PROJECTION topk2 AS SELECT EXTRACT(YEAR FROM d) d_year, i, d, f FROM tab1 LIMIT 1 OVER (PARTITION BYEXTRACT (YEAR FROM d), i ORDER BY d).

However, the user who creates the top-k projection does not see thecolumn EXTRACT(YEAR FROM d) in both the SELECT clause and PARTITION BYclause of the top-k projection creation statement. This implicitly addedexpression does not affect the partition and sort order of the top-kprojection. It is still partitioned on and ordered by column i.

FIG. 3 illustrates the execution of the following insert query: INSERTINTO tab1 VALUES (1, 10.0, 100, ‘c’, ‘cat’, ‘2014-01-20’). The columnsof input data are in the order of the columns in table tab1, which is[id, f, i, c, v, d]. The ExprEval operators below the data targets (DTs)of super projections, DT(sp1) and DT(sp2), redirect data to the columnorder of the projection columns of sp1 and sp2 respectively. TheExprEval operator below the data target of top-k projection, however,redirects data to a Top-K operator to compute a top-k function thatretrieves the top-k rows of values from each partition of rows of tabletab1. The resulting data from the top-k function is output to datatarget DT(topk1). The expression year(d) belongs to an implicitly addedcolumn, and split data into its year-based partition. The root operatorat the top of FIG. 3 may return a number of rows executed by theoperators below it.

As shown in TABLE 1 above, the input data from FIG. 3 may be bulk-loadedinto table tab1. The input data includes four rows in the example inTABLE 1. The actual data of the input data shown in TABLE 1 is stored inprojections sp1 and sp2. TABLE 4 shows the data of sp1 and sp2 after thefirst load. The columns shown in TABLE 4 are in the order of sp2, andsp1 has the columns ordered as i, c, v, d, f, id after the ExprEval forsp1.

TABLE 4 Storage Container id f i c v d 1 4 17 101 ‘b’ ‘bat’  2012-11-01’2 2 11 101 ‘d’ ‘dog’ ‘2014-01-15’ 1 10 100 ‘c’ ‘cat’ ‘2014-01-20’ 3 15100 ‘c’ ‘cow’ ‘2014-01-15’

TABLE 4 includes two different physical storage containers, eachrepresents a partition of the same year of column d.

TABLE 5 shows the top-k data of the first load for the top-k projectiontopk1 after the ExprEval operator and Top-K operator are executed forthe top-k projection. Even though the top-k projection is partitioned bysame values in column i, their actual PARTITION BY expressions are(year(d), i) as explained above. According to an example, all rows withthe same year(d) are in the same physical storage container.

TABLE 5 year(d) i d f 2014 100 ‘2014-01-15’ 15 2012 101 ‘2012-11-01’ 17

For top-k projections, according to an example, the top-k function isimplemented on the inserted data, and the pre-existing data in the top-kprojections is not combined with the inserted data. TABLE 6 shows thedata set of a second load (e.g., three rows of data).

TABLE 6 id f i c v d 5 20 100 ‘c’ ‘cat’ ‘2014-01-20’ 6 5 101 ‘d’ ‘dog’‘2014-01-15’ 7 25 100 ‘c’ ‘cow’ ‘2014-01-15’

For TABLES 7 and 8 illustrate the data stored in projections sp1, sp2,topk1 respectively after the second load. The boldface data is from thesecond load.

TABLE 7 Storage Container id f i c v d 1 4 17 101 ‘b’ ‘bat’  2012-11-01’2 2 11 101 ‘d’ ‘dog’ ‘2014-01-15’ 1 10 100 ‘c’ ‘cat’ ‘2014-01-20’ 3 15100 ‘c’ ‘cow’ ‘2014-01-15’ 5 20 100 ‘c’ ‘cat’ ‘2014-01-20’ 6  5 101 ‘d’‘dog’ ‘2014-01-15’ 7 25 100 ‘c’ ‘cow’ ‘2014-01-15’

TABLE 8 year(d) i d f 2014 100 ‘2014-01-15’ 15 2014 100 ‘2014-01-15’ 252014 101 ‘2014-01-15’  5 2012 101 ‘2012-11-01’ 17

As shown in TABLES 7 and 8, the data of the first and second load in thetop-k projection are not combined together. One benefit of this strategyis that the data storage system 100 does not have to read existing datawhile loading new data, but the side effect is the data stored in top-kprojection is not completely top-k. To make the partially top-k datainvisible to users, we run the top-k query or function again at a SELECTstep. For example, when a user selects data from the top-k projection,the top-k projection calculates the top-k rows for each partition of thetop-k projection and returns the result to the user. For example, if auser issues the following command: SELECT * FROM topk1, the actuallyplan executed looks like the plan of this query: SELECT i, d, f FROMtopk1 LIMIT 1 OVER (PARTITION BY i ORDER BY d); and the return data isshown in the two rows of TABLE 9 below.

TABLE 9 i d f 100 ‘2014-01-15’ 15 101 ‘2012-11-01’ 17

When a table has a top-k projection, users may be disallowed fromupdating its data. Similar to an update command, a delete command maynot be allowed on tables with top-k projections. However, users can dropthe whole partition of that table. As discussed above, the partitionexpression of the table by year may be implicitly added into the top-kprojection. Data of different partitions is stored in differentcontainers. Also, when a partition is removed (e.g., SELECT DROPPARTITION (‘tab1’, 2012), all of its corresponding containers areremoved without updating or touching other containers.

According to an example, each time data is inserted into a table, thedata is loaded into new files. After a while, there are many files whichcan slow down the reading process. To avoid too many files in the datastorage system 100, MERGE OUT merges data from different files of acolumn to one file. For a top-k projection, all the data is combinedagain during merge-out to determine the final merged result with top-kdata.

If a projection is created for a table that already has data, a refreshis done to copy appropriate data into the newly created projection. Ifthe new projection is a top-k projection, its refreshed data is eithercopied from an available top-k projection with the same definition orrecomputed from its table's super projection.

Data recovery for a top-k projection after a node in its cluster is downand brought back up is similar to the refresh process. The data can beeither copied from another top-k projection that is not affected by thenode failure or recomputed from its table's super projection. If thelatter happens, the top-k projection is recovered after its table'ssuper projections.

FIGS. 4-6 respectively depict flow diagrams of methods 400-600 for top-kprojections according to examples of the present disclosure. It shouldbe apparent to those of ordinary skill in the art that the methods400-600 represent generalized illustrations and that other operationsmay be added or existing operations may be removed, modified orrearranged without departing from the scopes of the methods 400-600.

With reference to FIG. 4, there is shown a flow chart of a method 400 toload data into a top-k projection, according to an example of thepresent disclosure. The method 400 is implemented, for example, by theprocessor 202 of computer 200 as depicted in FIG. 2.

In block 410, the load management module 110, for instance, may receivea load command to load input data including a plurality of rows ofvalues into columns of a database table. According to an example, theload command may include at least one of an insert command to load theinput data row by row and a copy command to load a set of rows in bulk.

In block 420, the load management module 110, for instance, may identifya top-k projection including at least one column of the database table.A column in the top-k projection may store values of results of a top-kfunction. According to an example, the top-k function may retrieve atop-k row from the input data for at least one of the columns of thedatabase table identified in the top-k function.

According to an example, the top-k projection may be identified byexamining a projection catalog. The projection catalog may storeinformation about a plurality of projections for the database table.Based on the information about the plurality of projections in theprojection catalog, the input data is directed to either the top-kprojection or another projection from the plurality of projections.

In block 430, the top-k function may be executed on the values of theinput data for each partition in the top-k projection. According to anexample, a partition is created to divide the plurality of rows based onsame values in a first column of the top-k projection. As discussedabove in TABLE 2, for instance, partitions are created for the samevalues of ‘100’ and ‘101’ in column i. The plurality of rows is thenranked within each partition according to values in a second column ofthe top-k projection. As discussed above in TABLE 2, for instance, theplurality of rows are then sorted according values in column d withineach partition ‘100’ or ‘101’. A top ranking row for each partition isthen calculated; and outputted to the top-k projection.

According to an example, the data storage system 100 sorts the data(e.g., the input data and the top-k data) in the top-k projection.Sorting includes ordering the rows in the projection according to acriteria. For instance, the criteria for the sorting may be specified inthe projection definition which may be used to create the top-kprojection. In other examples, the sort criteria may be predetermined orinput by the user outside the projection definition or provided byanother system. In addition, the data in a particular column of thetop-k projection may be sorted with respect to each or with respect todata in other columns. According to an example, the data in the top-kprojection may be sorted to enhance, e.g. maximize, performance ofqueries on the results data. By way of example, the data in the top-kprojection may be sorted to enable queries to run in a minimal amount oftime. Furthermore, the data in the top-k projection remains sortedaccording to the sorting initially performed on the data when it isloaded into the projection. For example, new data may be loaded into thetop-k projection. But the new data may not be combined with thepre-existing data in the top-k projection and then sorted. Instead, thepre-existing data remains in its previous sort order, and the new datais sorted without including the new data with the pre-existing data inthe top-k projection. The pre-existing data in the top-k projection maybe stored in a different physical container than the new data to keepthe data separated.

The data in the top-k projection may be encoded. For instance,run-length encoding may be performed on the data to compress it. Inother examples, other encoding techniques may be used. Also, the encodeddata in the top-k projection may be segmented. Segmenting may be basedon a hash function or another function to distribute data acrossdifferent nodes. In one example, the segmenting provides substantiallyeven distribution of the data in the projection across some or all ofthe nodes in a cluster for enhanced, e.g., optimal, query execution.Thus, the data in a top-k projection or any projection may be stored inone node or multiple nodes, and one or more copies of the data may bestored to provide redundancy in case of a failure of one of the nodes.

In block 440, the top-k row for each partition is stored in the columnsof the top-k projection. Also, as discussed above, new input data maynot be initially combined with data already stored in the top-kprojection. When a SELECT command is received for the top-k projection,the top-k function may be executed on all the data in the top-kprojection or on all the data in a partition in the top-k projection.Also, the data may be re-sorted. For example, all the data identified inresponse to the SELECT command may be sorted according to specified orpredetermined criteria.

With reference to FIG. 5, there is shown a flow chart of a method 500 tocreate and populate a top-k projection, according to an example of thepresent disclosure. The method 500 is implemented, for example, by theprocessor 202 of computer 200 as depicted in FIG. 2.

In block 510, the projection manager 120, for instance, may create atop-k projection. According to an example, the top-k projection maystore a top ranking row (i.e., a top-k row) for each partition in thetop-k projection. Once the top-k projection is created, the top-kprojection is then populated with top-k rows of data values.

For example, a set of columns may be selected from a database table. Theset of columns include a plurality of rows of values as shown in block520. According to a further example, an implicit partition column mayautomatically be added to the SQL statement that selects the set ofcolumns from the database table. This implicit partition column mayinclude the same values by which the database table was partitioned. Asdiscussed above the benefit of the implicit partition column is thatthis column may easily be removed from the top-k projection in responseto receiving a drop partition command to remove a partition of thedatabase table.

In block 530, the top ranking row (i.e., top-k) for each partition inthe top-k projection may be calculated. For instance, the rows of valuesmay be divided into partitions according to same values in a firstcolumn of the set of columns. The rows within each partition may then beranked according to values in a second column of the set of columns.Thus, the top ranking row for each partition may be outputted to thetop-k projection.

According to an example, the calculating of the top ranking row isperformed each time a user selects values from the top-k projection bymerging the sorted data. According to further examples, the method 500may receive a merge command to merge the plurality of rows of valuesfrom different files of a column in the database table and recalculate atop-ranking row for each partition in the top-k projection. The method500 may receive a refresh command to request a plurality of rows to in anewly created top-k projection, wherein the selected plurality of rowsis obtained by at least one of copying the requested plurality of rowsfrom a top-k projection with a same definition and recalculating theselected plurality of rows from a super projection of the databasetable. The method 500 may also receive a recover command to request aplurality of rows to store in a top-k projection of a recovered node ina database storage system, wherein the selected plurality of rows isobtained by at least one of copying the requested plurality of rows fromanother top-k projection with a same definition and recalculating theselected plurality of rows from a super projection of the databasetable.

With reference to FIG. 6, there is shown a flow chart of a method 600 toquery a top-k projection, according to an example of the presentdisclosure. The method 600 is implemented, for example, by the processor202 of computer 200 as depicted in FIG. 2.

In block 610, the query engine 130, for instance, may receive a top-kquery that requests the top-k rows of values from a database table. Atop-k projection that stores values of results of a top-k function maythen be identified. The top-k function may retrieve a top-k row for eachpartition in the top-k projection for at least one of the columns of thedatabase table identified in the top-k function. According to anexample, the identification of the top-k projection may be accomplishedby referencing a projection catalog that stores information about aplurality of projections for the database table, and includes user,schema, database table, and projection information.

At block 630, a determination is made as to whether the top-k projectionis identified from the plurality of projections for the database table.If the top-k projection is identified, then the top-k query is run onthe top-k projection, as shown in block 640. As discussed above, thetop-k query may select a set of columns from the database table, whereinthe set of columns include rows of values. The top-k query may thendivide the rows into a plurality of partitions according to same valuesin a first column of the set of columns, rank the rows within eachpartition according to values in a second column of the set of columns,calculate a top ranking row for each partition; and output the topranking row for each partition in the top-k projection. If the top-kprojection could not be identified, the top-k query may be run on asuper projection according to an example.

Accordingly, when a query is received that is based on the top-kfunction, the results can be quickly provided to the user from the top-kprojection. Thus, query execution performance is improved, i.e., faster.Execution of queries on the top-k projection may be faster than usingindexes and database tables to execute the queries if the projectionsare optimized for the queries.

What has been described and illustrated herein are examples of thedisclosure along with some variations. The terms, descriptions andfigures used herein are set forth by way of illustration only and arenot meant as limitations. Many variations are possible within the scopeof the disclosure, which is intended to be defined by the followingclaims—and their equivalents—in which all terms are meant in theirbroadest reasonable sense unless otherwise indicated.

What is claimed is:
 1. A data storage system comprising: a data storageto store a plurality of projections associated with a database table;and at least one processor to: receive a load command to load input dataincluding a plurality of rows of values into columns of the databasetable, the plurality of rows of values including a plurality ofpartitions; identify a top-k projection including at least one column ofthe database table; execute a top-k function on the input data todetermine a predefined number of top ranking rows of values for each ofthe plurality of partitions; store the predefined number of top rankingrows of values for each of the plurality of partitions in at least onecolumn of the top-k projection; receive a top-k query requesting top-krows of values among the plurality of rows of values from the databasetable; run the top-k query on the top-k projection to retrieve thepredefined number of top ranking rows of values for each of theplurality of partitions in the top-k projection responsive todetermining that the top-k projection is associated with the top-kquery; run the top-k query on a super projection responsive todetermining that the top-k projection is not associated with the top-kquery; and return query results responsive to running the top-k query.2. The data storage system of claim 1, wherein to execute the top-kfunction, the at least one processor is to: create each of the pluralityof partitions, wherein the plurality of partitions divide the pluralityof rows of values based on same values in a first column of the top-kprojection; rank the plurality of rows of values within each partitionaccording to values in a second column of the top-k projection;calculate the predefined number of top ranking rows of values for eachof the plurality of partitions; and output the predefined number of topranking rows of values for each of the plurality of partitions in thetop-k projection as values of results of execution of the top-k functionfor storing in the top-k projection.
 3. The data storage system of claim1, wherein to identify the top-k projection, the at least one processoris to: examine a projection catalog that stores information about theplurality of projections; and direct the input data to at least one ofthe top-k projection or another projection based on the informationabout the plurality of projections in the projection catalog.
 4. Thedata storage system of claim 1, wherein the at least one processor isto: sort results of the execution of the top-k function in the top-kprojection; receive a second load command to load new input data intothe database table; execute the top-k function on the new input data andnot on data stored in the top-k projection prior to receiving the newinput data, wherein the data stored in the top-k projection prior toreceiving the new input data remains ordered according to the sort;store results of the execution of the top-k function on the new inputdata in the top-k projection; and sort the results of the execution ofthe top-k function on the new input data in the top-k projection.
 5. Thedata storage system of claim 1, wherein the load command includes atleast one of an insert command to load the input data row by row or acopy command to load a set of rows in bulk.
 6. The data storage systemof claim 1, wherein the at least one processor is to: determine apredetermined limit corresponding to the predefined number of topranking rows of values for each of the plurality of partitions to bestored in the at least one column of the top-k projection, thepredetermined limit corresponding to a limit defined in the top-k queryassociated with the top-k projection.
 7. The data storage system ofclaim 1, wherein the at least one processor is to: create the superprojection to include all columns of the database table; and direct theinput data to be loaded to the top-k projection and/or the superprojection based on descriptions of the plurality of projections in aprojection catalog, wherein the at least one processor loads the inputdata to the super projection for execution of queries that are notassociated with the top-k projection.
 8. A method to create and populatea top-k projection and run a top-k query, comprising: creating, by aprocessor, a top-k projection to store results of execution of a top-kfunction that is associated with the top-k projection; selecting a setof columns from a database table, wherein the set of columns include aplurality of rows of values; calculating a predefined number of topranking rows of values for each partition of a plurality of partitionsin the top-k projection, wherein the calculating of the predefinednumber of top ranking rows of values includes: dividing the plurality ofrows of values into the plurality of partitions according to same valuesin a first column of the set of columns; ranking the plurality of rowsof values within each of the plurality of partitions according to valuesin a second column of the set of columns; and outputting the predefinednumber of top ranking rows of values for each of the plurality ofpartitions to be saved in the top-k projection; receiving a top-k queryrequesting top-k rows of values among the plurality of rows of valuesfrom the database table; running the top-k query on the top-k projectionto retrieve the predefined number of top ranking rows of values for eachof the plurality of partitions in the top-k projection responsive todetermining that the top-k projection is associated with the top-kquery; running the top-k query on a super projection responsive todetermining that the top-k projection is not associated with the top-kquery; and returning query results responsive to running the top-kquery.
 9. The method of claim 8, wherein the calculating of thepredefined number of top ranking rows of values is performed each time auser selects values from the top-k projection by merging sorted data.10. The method of claim 8, wherein selecting the set of columns from thedatabase table includes: adding an implicit partition column to theselected set of columns from the database table, wherein the implicitpartition column includes same values by which the database table waspartitioned.
 11. The method of claim 10, wherein the implicit partitioncolumn is removed from the top-k projection in response to receiving adrop partition command to remove a partition of the database table. 12.The method of claim 8, further including: receiving a merge command tomerge the plurality of rows of values from different files of a columnin the database table; and recalculating the predefined number of topranking rows of values for each partition in the top-k projection. 13.The method of claim 8, further including: receiving a refresh command torequest a plurality of rows of values to be refreshed in a newly createdtop-k projection, wherein the plurality of rows of values to berefreshed is obtained by at least one of copying the plurality of rowsof values to be refreshed from a top-k projection with a same definitionor recalculating the plurality of rows of values to be refreshed fromthe super projection of the database table.
 14. The method of claim 8,further including: receiving a recover command to request a plurality ofrows of values to store in a top-k projection of a recovered node in adatabase storage system, wherein the requested plurality of rows ofvalues to store are obtained by at least one of copying the requestedplurality of rows of values to store from another top-k projection witha same definition or recalculating the requested plurality of rows ofvalues to store from the super projection of the database table.
 15. Themethod of claim 8, further comprising: determining a predetermined limitcorresponding to the predefined number of top ranking rows of values foreach of the plurality of partitions to be stored in the set of columns,the predetermined limit corresponding to a limit defined in the top-kquery associated with the top-k projection.
 16. The method of claim 8,further comprising: creating the super projection to include all columnsof the database table; and populating data in the top-k projectionand/or the super projection based on descriptions of the top-kprojection and/or the super projection in a projection catalog, whereinthe data is populated in the super projection for execution of queriesthat are not associated with the top-k projection.
 17. A non-transitorycomputer readable medium to query a top-k projection, including machinereadable instructions executable by a processor to: receive a loadcommand to load input data including a plurality of rows of values intocolumns of a database table, the plurality of rows of values including aplurality of partitions; identify a top-k projection including at leastone column of the database table; execute a top-k function on the inputdata to determine a predefined number of top ranking rows of values foreach of the plurality of partitions; store the predefined number of topranking rows of values for each of the plurality of partitions in atleast one column of the top-k projection; receive a top-k queryrequesting top-k rows of values among the plurality of rows of valuesfrom the database table; identify the top-k projection associated withthe top-k query; run the top-k query on the top-k projection to retrievethe predefined number of top ranking rows of values for each of aplurality of partitions in the top-k projection when the top-kprojection associated with the top-k query is identified; run the top-kquery on a super projection when the top-k projection associated withthe top-k query is not identified; and return query results responsiveto running the top-k query.
 18. The non-transitory computer readablemedium of claim 17, wherein to identify the top-k projection associatedwith the top-k query, the machine readable instructions are executableby the processor to: reference a projection catalog that storesinformation about a plurality of projections for the database table,wherein the information includes user, schema, database table, andprojection information.
 19. The non-transitory computer readable mediumof claim 17, wherein to run the top-k query, the machine readableinstructions are executable by the processor to: select a set of columnsfrom the database table, wherein the set of columns include theplurality of rows of values; divide the plurality of rows of values intothe plurality of partitions according to same values in a first columnof the set of columns; rank the plurality of rows of values within eachof the plurality of partitions according to values in a second column ofthe set of columns; calculate the predefined number of top ranking rowsof values for each of the plurality of partitions; and output thepredefined number of top ranking rows for each of the plurality ofpartitions to be saved in the top-k projection.
 20. The non-transitorycomputer readable medium of claim 17, wherein the processor is to:determine a predetermined limit corresponding to the predefined numberof top ranking rows of values for each of the plurality of partitions tobe stored in the at least one column of the top-k projection, thepredetermined limit corresponding to a limit defined in the top-k queryassociated with the top-k projection.